1) Volumetric Displays—Why Are They Attractive?
Two-dimensional (2-D) display technologies are pervasive and include all manner of CRTs, LCDs, plasma flat panels, more recently OLEDs, and even projected images such as movie screens or slide-show images. They are utilized in everything from televisions, PDAs, PCs, instruments, wrist-watches, appliances, automobiles, digital still and video cameras, video playback devices, and cell-phones. Many of these are capable of presenting static and/or moving imagery and/or alphanumeric messages. They will be with us indefinitely. However, they are not ideal for the presentation, manipulation of, or navigation through complex spatial scenes and data.
As a first example, CAD (computer-aided design) engineers utilize software on their CAD workstations to section and rotate their work on a 2-D screen. This approach is adequate but not natural. CAD engineers almost always have parts made in order to handle them and get the complete picture of form and fit, if not also of function. In fact, one could argue that the stereolithography market, which provides solid computer-generated polymeric CAD prototypes, is a direct response to the inadequacy of 2-D CAD displays. True 3-D volumetric displays would be very, very helpful.
As a second example, air-traffic controllers utilize a 2-D screen on which the third dimension, altitude, is not physically represented by a display dimension. Because of this, the controllers have to utilize alphanumeric altitude tags juxtaposed to the moving blips (planes). In essence and by default, the controllers memorize the altitude data on these tags, which limits how much air traffic they can handle. Although alarms can be set for “unacceptable” altitudes, one wishes to avoid responding to a stream of such alarms. This is a perfect application for a true 3-D display.
A third example would be electronic games which require the user(s) to frequently switch between views of the game, an approach well short of having a simpler 3-D overview viewable from any perspective. This is a large market that would welcome this superior visualization capability. The game and gaming markets push the forefront of graphics.
A fourth example would be for surgeons who would be greatly enabled by having true 3-D imagery of their patient's anatomy to work on and navigate through. 3-D displays could substantially simplify training of surgeons and the quality control and preciseness of minimally invasive manipulations.
A fifth example would be the two independent fields of molecular engineering and nanotechnology, which are fundamentally limited by the difficulty of visualizing and modeling the complex fit and interactions between highly irregularly-shaped molecules, proteins and/or atoms or assemblages thereof. This would be greatly simplified with true 3-D displays viewable from any perspective.
The 50+ year-old 2-D display paradigm offers a “window” through which one can look at a particular view of something. It does not satisfy all of the many depth-cues that the real objects would. The objects are always “outside” the window and the viewer is always inside. The ability to move one's head (or walk around) and see a variety of perspective views is a natural means of dealing with spatial complexity in the quickest and most efficient manner. A truly 3-D display should get rid of the “window” analogy and create objects that have immediate local presence “on your desk” rather than views of objects that are removed from the viewer.
Thus, it is no surprise that the history of attempts to design and sell truly 3-D displays is a very long and very rich one, revealing amazing creativity and persistence. However, even today, such displays are not yet a significant market success for a variety of reasons we will discuss below. There is zero doubt that a practical affordable 3-D display with solid-like visualization abilities would have a large and growing market, likely initially in scientific/engineering/medical visualization and serious games and gaming. Surely part of the reason for the limited commercial success is because it is correctly recognized that the design of a realistic 3-D display is a very difficult multidisciplinary engineering problem. However, as for any technical field, new approaches can break down old barriers. We herein offer some new approaches that we believe circumvent many prior art problems and make practical 3-D displays a tractable market challenge.
2) Volumetric Displays—What Are the Attributes of a Good Volumetric Display and Which Have Been Difficult to Achieve?
The best definition we have found is from the only book dedicated to the subject, “Volumetric Three Dimensional Display Systems” by Blundell and Schwarz (Wiley-IEEE Press, Mar. 3, 2000, ISBN #0471239283). It defines a volumetric 3-D display in this manner: “A volumetric display device permits the generation, absorption, or scattering of visible radiation from a set of localized and specified regions within a physical volume”. A significant point here is that the volumetric pixels, known in the 3-D art as voxels, may produce optical contrast by any one or more of: a) the production of light, b) the scattering of light, or c) the partial or complete blocking of light. For an ideal 3-D display of this type, one or more observers may look at the displayed objects or scenes from multiple moving or static perspectives and have the objects or scenes appear as solid or semisolid objects that exhibit occlusion effects. Just as for 2-D displays, a minimum refresh frame rate is required, and this has empirically been found to be about 20 to 25 refreshed frames per second for volumetric displays and ideally should be even higher. As for resolution or voxel spacing, it is desirable that the displayed objects or scenes are accurately depicted without aliased or choppy edges, so the required resolution is a function of the detail to be imaged. In general, prior art 3-D work has mostly involved voxels with on the order of 1 mm (minimum) or so spacing, which is 2× to 4× coarser than modern 2-D displays but sufficient for some tasks in 3-D.
The challenge of going from 2-D to 3-D is that the number of pixels goes from N×N (N-squared) for 2-D pixels to N×N×N (N-cubed) for 3-D voxels assuming equivalent resolution. In general, the size of the data pipe (bandwidth) needed to build a large 3-D display with significantly better than 1 mm voxel spacing has until recently been prohibitive. Recently, however, means to construct multigigabit per second image-updating data paths are becoming available. So now it is more the poor image quality and cost that are becoming the issue, rather than the brute force processing power. Our inventive contributions herein are particularly enabled by our disclosed advances in contrast generation mechanisms that overcome many of the remaining image quality shortcomings.
Prior art 3-D displays generally presented sequentially illuminated slice-at-a-time imagery. That is, the object or scene being imaged was cyclically illuminated in space one slice (or sub-frame portion) at a time and in a manner fast enough that the eye “sees” an overall fused object that is dimly lit at an apparent “average” pseudo-constant intensity. The inability of electronics to “keep up” with a real 3-D scene by updating it all (all sub-frame slices in a full frame depiction) simultaneously is hidden from the viewer because the eye and brain “average out” and fuse together what it sees via the known fusion-persistence effect. Thus, rapid sequential slice updates are seen as illuminated objects or glowing surfaces which are presented “just fast enough” to fool the observer into thinking that the object is illuminated all the time.
However, prior art displays have suffered in image quality for a number of reasons. First have been the known dead zones, which are areas of the image volume which have either no active voxels, obscured voxels or distorted misshapen voxels. This is because of reasons pertaining to the varying geometry of propagation of radiation (light, electrons, etc.) toward a target voxel on a moving solid projection screen. A second reason is because if a moving solid screen technique is used (see below), the screen needs a drive shaft and a stiffening frame and the screen itself cannot be totally transparent and smooth or it will not work. A third reason, applicable to static and moving screen displays, is that electrical connections to interior voxels, if necessary, cannot be made totally transparent and totally non-refracting. A fourth reason is that nobody has ever been able to provide a contrast mechanism that has high optical contrast between the ON and OFF states (when viewed from outside the assembled operating display). As an example, a liquid crystal voxel which turns dark and then clear does not really turn totally clear. Building a static (no moving parts) 3-D display by stacking or juxtaposing hundreds of not-quite-clear voxels and not-quite-clear and refracting interconnects and antireflection films results in a semi-opaque haze even when no object is displayed. Finally, many attempts have involved stacking of panels in the third dimension, each such panel in essence being a 2-D display. It is extremely difficult to make the interfaces between such stacked panels totally invisible, especially if electrical conductors such as semi-transparent and refracting indium-tin oxide are employed. A last reason has been mediocre resolution (large voxel spacing of one or more millimeters) due to graphics engine limitations. Only recently have affordable graphics engines provided the multigigabit per second performance that is required even for the far from detailed nonideal 1 mm voxel resolution.
Many prior art 3-D display attempts have also resulted in excessively noisy or excessively heavy displays. We, however, can somewhat excuse the “heavy” criticism as even CRTs of high quality and high volume are clearly “heavy” yet have for decades been in high demand. However, a 3-D volumetric display which comprises a solid block of stacked LCD plates is going to be much heavier than even a vacuum CRT. At least a CRT is hollow.
A display that makes a lot of noise is not excusable in most applications. Thus, a heavy 3-D display (as heavy as a CRT) that otherwise works on the order of as well as a 2-D CRT in image quality will certainly have a market, presuming it does not have a substantially equivalent lighter 3-D competitor.
We should note, per the above definition of a volumetric 3-D display, that there are a huge number of “3-D” displays which involve vibrating mirrors, lenticular lenses, crossed-polarizers, eye-switching viewing glasses, etc., all of which fool the eye or brain into thinking there is a third dimension present when there is not. These are more properly termed 2.5-D displays, as the voxels are not physically spatially distributed. Most of these only work from one sitting position. Many of these can give users fatigue, eyestrain or a headache. We will discuss only 3-D displays that truly physically place voxels into three dimensions (for at least transitory moments) such that they can be visualized in their true light-emitting, light scattering or light-absorbing positions from multiple perspectives.
So, in summary, a 3-D display should ideally be as clear (or at least featureless) as glass when it is off and when it is on it should not be subject to haze or fog surrounding the image objects. One should have a bright high-contrast image that, preferably, does not require a severely darkened room in order to see it. Ideally, but not necessarily, would be an ability of the display to depict occlusion (visual obstruction of objects behind other objects). In this manner, solid-like opaque objects would be seen as opposed to ghost-like semitransparent or transparent objects. Our invention herein offers a degree of such occlusive ability. A weight equivalent to, or preferably lighter than, that of a vacuum CRT is preferred. Finally, it has been clearly established that 2-D image-projector based (moving screen) 3-D displays are much more able to keep up with the required image refresh rates than scanned-spot 3-D displays. Thus, the display will preferably be illuminating significantly more than one voxel at a time per beam, if a beam is used at all. This is a preference for good bandwidth efficiency and to get better refresh rates and not necessarily a hard requirement.
3) Volumetric Displays—What Approaches Have Been Tried and What Are the Advantages and Drawbacks of Each Type?
Static Volume—No Moving Parts
A static volume 3-D display typically comprises a stack of 2-D image display panels which is at least partially transparent in stacked form. One can operate it in one of three ways: a) a moving (via electrical switching) screen is formed inside the stack upon which appropriate image slices are optically projected from one end, b) a moving (via electrically switching) screen is illuminated using its own in situ emitters to show the appropriate image slice (no projection), or c) a moving slice (or subset of voxels) is illuminated using an external light source, the slice or subset pixels being electrically rendered opaque or transparent according to the appropriate slice of the image, but the illumination not necessarily comprising a projected image. An alternative approach is to turn on all of the voxels that represent the 3-D object at one time at a frame rate. This has severe signal bandwidth limitations but would certainly be preferred in order to offer some occlusion effect. It has not yet been achieved.
Intrinsic advantages of static 3-D displays (seen to date only as paper designs) would seem to include: 1) reliability, 2) silent operation, 3) ease of defining boxlike display volumes, and 4) acceptable bandwidth for moving slice modes.
Intrinsic disadvantages include: 1) haze or lack of transparency due to voxel interconnects and electrodes as well as stacked-panel antireflection or index-matching films, 2) optical distortions due to mismatching optical indices at stack interfaces, 3) weight much greater than for CRTs of equivalent dimension, 4) highest probable cost, and 5) dim translucent imagery incapable of opacity, occlusion and shadowing (moving slice mode).
Some examples of static displays from the prior art are:                U.S. Pat. No. 4,670,744 to Buzak entitled “Light reflecting Three Dimensional Display System”. This is a stacked LCD display approach.        U.S. Pat. No. 5,745,197 to Leung et al entitled “Three Dimensional Real-Image Volumetric Display System And Method”. This is a stacked panel LCD approach, for example, typically using external illuminators.        U.S. Pat. No. 5,764,317 to Sadovnik et al entitled “3-D Volume Visualization Display”. This is a stacked LCD panel approach wherein an image projector projects image slices upon opaque activated slices in rapid sequence.        U.S. Pat. No. 5,813,742 to Gold et al entitled “Layered Display System And Method For Volumetric Presentation”. This is another LCD stack, but herein they form a hemisphere rather than a block. An interior projector illuminates one opaque layer at a time as above.        U.S. Pat. No. 5,929,572 to Whitesell entitled “Electroluminescent Arrays Layered To Form A Volumetric Display”. This is a stacked panel display like the above but herein the panels and voxels comprise self-emitting light sources rather than opaque regions, which need to be externally illuminated.        U.S. Pat. No. 6,100,862 to Sullivan entitled “Multi-Planar Volumetric Display System And Method Of Operation”. This is another stacked LCD panel and projector system for illuminating the moving opaque or switched slice.        
Swept-Volume—Rotated or Translated Screens—Moving Physical Parts
A prior art swept-volume 3-D volumetric display typically utilizes a mechanically-moving, physically-solid screen or layer. Appropriate sequential slice images are projected onto or activated or written upon (or within) the moving screen when it is at each corresponding sequential spatial position. By “moving” we mean rotation, but translation has also been tried to a much lesser extent. Solid translating screen displays will probably never be as easy to make and operate as rotating screens; they have inertia operating against them. The major subdivision within swept-volume rotating displays is between passive screens and active screens, i.e., screens that reflect (or block) light vs. screens that produce light (by phosphorescence of the voxel due to an e-beam impinging upon it or by self-emission from a powered screen-mounted LED, for example).
A passive screen could be, for example, a translucent screen for acceptance of a projected optical light image or illuminating scanned laser spot. In other words, a passive screen does not produce its own optical emissions; rather, it redirects them to (or away from) the eyes from elsewhere.
Intrinsic advantages of a swept passive screen include: 1) lowest cost, 2) light weight, and 3) elimination of the complexity of voxel interconnects required for an active screen.
Intrinsic disadvantages of a swept passive screen include: 1) dead zones due to difficult geometries of impingement of the illumination (or activation) beam upon the screen or due to rotation shafts or screen-stiffening frames, 2) noisy operation, 3) reliability, and 4) dim translucent imagery incapable of opacity, occlusion and shadowing.
Examples of rotated passive screens from the prior art include:                U.S. Pat. No. 4,983,031 to Solomon entitled “Three Dimensional Volumetric Display System”. This display has a passive screen which rotates and upon which an image (slice by slice) is projected. Some such rotating screen and projector systems have some or all of the optics co-rotating with the screen (see below) (this reference has just some of the optics rotating with the screen). Other references have stationary projectors off-board the rotating screen.        U.S. Pat. No. 5,042,909 to Garcia entitled “Real time Three Dimensional Display With Angled Rotating Screen And Method”. This display has laser beam spots scanning a tilted screen. Recall the slow frame rate issues of any prior art scanned-spot system.        U.S. Pat. No. 5,082,350 to Garcia et al entitled “Real Time Three Dimensional Display System for Displaying Images In Three dimensions Which Are Projected Onto A Screen In Two Dimensions”. This display is very similar that that of U.S. Pat. No. 5,042,909 above.        U.S. Pat. No. 5,172,266 to Garcia et al entitled “Real Time Three Dimensional Display”. This is yet another rotating or compound rotation passive screen.        U.S. Pat. No. 5,854,613 to Soltan et al entitled “Laser Based 3-D Volumetric Display System”. This device has multiple laser beams impinging upon a rotating passive helical screen to form 3-D images.        U.S. Pat. No. 5,936,767 to Favalora entitled “Multiplanar Autostereoscopic Imaging System”. This display describes a number of images projected upon rotating passive screens.        U.S. Pat. No. 5,954,414 to Tsao entitled “Moving Screen Projection Technique For Volumetric Three-Dimensional Display”. This is one of the first patents involving projection of images on moving passive screens of both the rotating and reciprocating or translating type.        U.S. Pat. No. 6,064,423 to Geng entitled “Method And Apparatus For High Resolution Three Dimensional Display”. This display uses a spatial light modulator to project images upon a rotating helical screen. The text has a nice prior art overview for the interested reader. The taught spatial light modulator or SLM is a means of image projection as opposed to building up images using scanned spots.        U.S. Pat. No. 6,302,542 to Tsao entitled “Moving Screen Projection Technique For Volumetric Three Dimensional Display”. This display is similar to that of U.S. Pat. No. 5,954,414 above.        
An active rotating screen could, for example, comprise a phosphor-coated screen which generates light emission at points whereupon a steered and gated off-board CRT electron-beam(s) impinges. This approach is like a TV picture tube, but with a moving phosphor layer or screen. Relative to a passive screen, an active screen produces its own light (or contrast) as by phosphor excitation of a screen phosphor or by selective switched illumination of a screen-mounted light-emitting diode (LED) or solid-state or gas laser. Historically, scanned beam active screen volumetric displays have had low frame rates or have resulted in sparse images. This is because scanned beam (scanned spot) systems typically only have the scanning bandwidth to utilize about 1% of the available voxels for any reasonable screen rotation rate. (This, however, is no longer true if entire 2-D images are projected as opposed to individual image spots or voxels making up such images. For that a passive screen suffices.)
Intrinsic advantages of a swept active screen include: 1) minimization of or elimination of image projection and associated optics (however, it must be replaced with complex and costly switching means), 2) elimination of the complexity of voxel interconnects as used in static displays (for an e-beam impinged active phosphor screen), and 3) light weight.
Intrinsic disadvantages of a swept active screen include: 1) dead zones due to difficult geometries of impingement of the illumination beam upon the screen or due to rotation shafts or screen-stiffening frames, 2) noisy operation, 3) reliability, and 4) dim translucent imagery incapable of opacity, occlusion and shadowing.
Examples of rotated active screens from the prior art include:                U.S. Pat. No. 4,160,973 to Berlin, Jr., entitled “Three Dimensional Display”. Therein are described a variety of self-illuminated moving screen displays, using, for example, LEDs.        U.S. Pat. No. 5,703,606 to Blundell (an author of the 3-D display book we referenced above) entitled “Three Dimensional Display System”. This patent describes phosphor-coated screens impinged by up to three e-beams in a vacuum vessel.        U.S. Pat. No. 6,054,817, also to Blundell, entitled “Three Dimensional Display System”. This patent describes algorithm and beam-detection improvements applicable to e-beam/phosphor screens.        U.S. Pat. No. 6,115,006 to Brotz entitled “Rotating Display Device And Method For Producing A Three-Dimensional Real Image”. This display utilizes an active spiral or helical screen having LEDs or field-emission light emitters.        WO 01/78410 A3 to Favalora et al entitled “Projection Screen For A Multiplanar Volumetric Display”. This reference covers some basic features of rotating screens for volumetric displays.        
Co-Rotating Projection Displays, Crossed Beam Displays
This category covers those that do not clearly (or only) fall into one of the above categories. The best example is rotating swept screens wherein all of the image projection means are rotated with the screen itself; thus, the geometrical relationship between projector and screen is constant and has minimal or no dead zones (dead zones due to variable geometry image projection). This approach solves many of the dead zone issues; however, some issues still exist since a rotating screen needs a shaft and may need a stiffening frame. Such a “co-rotated” projector/screen may likely also need a sophisticated slip-ring (sliding contacts) assembly to pass signals to and from the rotating portions. This can be done technically, but cost and reliability are the question marks. This approach is most like a swept volume approach; however, the portion being rotated comprises a static (to itself) screen and projector subsystem. Examples of such systems from the art include:                U.S. Pat. No. 4,983,031 to Solomon entitled “Three Dimensional Volumetric Display System”. This device was already mentioned under the above passive rotation section but we mention it here because it was one of the first to utilize co-rotating optical projection means.        U.S. Pat. No. 5,148,310 to Batchko entitled “Rotating Flat Screen Fully Addressable Volume Display System”. This is another co-rotating screen and projection means approach.        U.S. Pat. No. 5,678,910 to Martin entitled “Multiple Angle Projection For 3-D Imagery”. This is another multiprojector/screen system with co-rotation.        U.S. Pat. No. 6,183,088 to Lore et al entitled “Three Dimensional Display System”. This is from Actuality Systems, Inc., which has a product on the market of this type. This patent covers tapering of the rotating screen edges to reduce dead-zones and dark-lines. Actuality Systems, Inc. uses a co-rotating screen and projection optics. They utilize digital light mirror (DLM™) chips as used in video projector products to get gigabit per second and higher bandwidths.        U.S. Pat. No. 6,487,020 to Favalora entitled “Volumetric Three Dimensional Display Architecture”. This is a twist on co-rotating screen/projection systems wherein the screen comprises a lenticular screen.        U.S. Pat. No. 6,554,430 to Dorval et al entitled “Volumetric Three Dimensional Display System”. This patent seems to best describe the Actuality Systems, Inc. product mentioned above. Their rotating screen is translucent and they utilize DLMs or digital light mirrors to advantage in a co-rotation subassembly. Since this is the only product on the market commercially, we have to consider it as the best prior art.        
It is an appropriate point here to reference two papers discussing the Actuality Systems, Inc. co-rotated projector product as follows:                1) Favalora, Gregg et al, “100 Million-Voxel Volumetric Display”, April 2002, Society Of Photo-Optical Instrumentation Engineers/SPIE proceedings. Further details beyond the above Actuality Systems, Inc. patents are explained herein. The basic Actuality Systems, Inc. Perspecta™ product's rotating screen affords 768×768 resolution for each of the 198 slices per 360 degree rotation. The display rotates at 730 rpm and comprises a translucent screen. A Texas Instruments Digital Light Mirror (DLM™) system comprises three such DLM chips for red, green, and blue. A rotating folded compound mirror system projects the DLM images at a 45 degree angle upon the rotated screen. The image volume comprises roughly most of a 10-inch diameter rotating sphere which is situated within a second static sphere. The system utilizes about 3 to 6 Giga-bytes of SRAM, which is of double buffered, double data-rate design. Color is dithered so that the inherent 8 color capability appears to have hundreds of colors.        
Without doubt, there will be some market for this product and its follow-on products. Given that we intend to improve upon the performance that the Actuality Systems, Inc. product demonstrates, we now point out its primary drawback compared to an “ideal” volumetric display. A simple look at FIGS. 3, 7, and 9 of the foregoing reference will make it clear that the displayed solids or subjects have the appearance of being immersed in a thick fog and being ghost-like in appearance, even without the fog surrounding (and overlying) the images. Despite this, moderate detail can be made out. There are, however, no test results provided to demonstrate what the actual achieved resolution or achieved contrast is for their “solid” images. We also note FIG. 9 of the foregoing reference again, wherein one can see that the 10-inch image volume sphere is surrounded by a 24 inch diameter static sphere. The need for the second spherical enclosure is not explained but we deduce that the inner sphere rotates and the outer sphere does not. This may be so as to eliminate direct drag on the screen itself and reduce that to a problem of drag on a sphere rotating within its own confines, which is a much quieter and reduced power situation. One of us (JWS) has personally viewed the product in the last year and these images seem consistent with what was personally experienced in a hands-on viewing at a display symposium. It should also be noted that the outer sphere certainly adds some undesirable optical effects whose magnitude are unknown to us.                2) Favalora, Gregg et al, “Spatial 3-D: The End of Flat Screen Thinking”, Actuality Systems, Inc. White Paper #1 of 3. This recent paper primarily argues the benefits of spatial visualization over flat-screen visualization as proven by several US Navy studies. Use of the known DICOM, OPEN GL and VRML image format standards are described for medical, CAD and other interactive spatial work. No display details are discussed therein.        
Yet another group of displays involve selectively “activating” points in the display volume as by overlapping two laser or e-beams. The volumetric media requires both beam energies to be incident in order for the voxel to emit light. The points may be spatial locations in a laser-activatable photonic crystalline glass or emission gas or may be particles that are suspended at such points and illuminated as by a laser. To our knowledge, the best of these are the photoemitting crystal block display which has its voxels activated by the cooperative action of crossed laser beams. However, it has been found exceedingly difficult to scale such technology beyond a few cubic centimeters for materials and other reasons. Such units are also very very bulky and very expensive because the required lasers are bulky.
Example of these types of “crossed-beam” displays include:                U.S. Pat. No. 3,829,838 to Lewis et al entitled “Computer Controlled Three-Dimensional Pattern Generator”. This is a basic crossed beam display. We are not aware of a prototype display.        U.S. Pat. No. 4,063,233 to Rowe entitled “Three Dimensional Display Devices”. This display uses crossed e-beams in a “phosphor dust cloud”, the dust cloud also being stirred in a novel manner. We have never seen a working prototype of a phosphor dust display, but we suspect that three difficult issues would arise as follows: 1) the dust that is not being activated will still scatter (or block) substantial light causing fog, 2) the dust particles may be substantially moved by the beam or beam-interactions adding unwanted effects, and 3) inter-particle interactions and particle-wall interactions with and without electrical activation, such as clumping. Similar devices try to use a gas (instead of dust), such as rubidium vapor, to overcome these problems; however, these gas approaches have other problems such as blooming of the voxel. We are not aware of any functional prototypes of any crossed-beam gas systems. At a minimum, a gas-based system has the advantage of transparency of the unactivated gas.        U.S. Pat. No. 4,881,068 to Korevaar et al entitled “Three Dimensional Display Apparatus”. This display uses crossed laser beams in a responsive gas image volume. We know of no working prototypes.        U.S. Pat. No. 5,627,554 to Jefferson entitled “Segmented Direct Volume Display Device And Method”. This display indeed describes crossed beam activation of gases and even of a gel. We know of no working prototypes.        U.S. Pat. No. 5,684,621 to Downing entitled “Method And System For Three Dimensional Display Of Information Based On Two-Photon Up-Conversion”. This is one of the best known and prototyped crossed beam displays using a photoactive block of material as the display volume. It is a clever approach; however, it has been very very difficult to make a sufficiently large display (a large crystal is needed) and to find a way to implement the required lasers and scanners compactly. It works for a very very small prototype crystal which is, even then, dwarfed by its external lasers and scanners. So far, it is a lab success only, and on a very small size scale at that.d) Fountains and Other Liquid or Bubble Aesthetic and Amusement Devices        
We would not be complete if we did not mention water-fountain based amusement and alphanumeric “displays” or, more correctly per their teaching, “aesthetic and amusement devices”. Most of these are very very large, but a few are breadbox sized. What they all have in common is that they utilize static or very slow moving water droplets or air bubbles in water or glycerin, akin to common fountains and water displays. A requirement of any 3-D volumetric graphical display is fast frame rate (e.g., 24 to 30 frames per sec minimum) and good resolution (e.g., probably one or more voxels per linear mm in three dimensions, depending on fineness of detail needed). It is readily apparent that these prior art decorative and amusement devices, albeit creative, useful and pleasant, could never provide the frame rate and resolution needed for our application. This is because the contrast phenomenon in them (shape of a water jet or location of a water droplet) all move at very low velocities to avoid becoming a random spray or aerosol in air. Their already complicated plumbing could not be compacted without going to a new plumbing paradigm. They also cannot emit small enough droplets fast enough, densely enough, or with spatial control anywhere near what is needed for our invention. Viewed in terms of our invention, thousands, if not millions, of droplets need to be created in space very rapidly and very close together for moving images, without appreciable unintended motions. (For example, if two neighboring droplets transit 12 inches of our display volume, they start with a 0.5 mm spacing and preferably end with a 0.5 mm spacing.) This amounts to creating a semi-ordered fog with a three-dimensional ordering of significant degree. One cannot simply turn the pressure up on a fountain and make the drops go faster without creating a fine turbulent spray and a disordered mess (fog plume or spray plume). One also needs powerful graphics processing and communication capabilities which are not suggested. None of these devices meet or suggest our purposes.
Examples of such amusement and aesthetic devices include:                U.S. Pat. No. 3,387,782 to Mizuno entitled “Apparatus For Producing A Fountain Including A Stroboscopic Light”. Although not a 3-D fountain, the inventor utilizes dynamically warping and deforming sheets of cascading water and lights these films using stroboscopic lighting to freeze periodic phenomenon. It is an aesthetic device.        U.S. Pat. No. 4,094,464 to Kawamura et al entitled “Three Dimensional Display Device using Water Fountain”. This device is again not a true 3-D fountain but a 2.5-D fountain for presentation of alphanumerics which are either unchanging or very slowly changing and of large size.        U.S. Pat. No. 4,265,402 to Tsai entitled “Strobed Liquid Display Device And Head Therefor”. This device could be described as an aesthetically pleasing showerhead-type fountain capable of emitting a helical shaped stream or sheet of droplets. Alphanumerics are not taught; this is a decorative or amusement device.        U.S. Pat. No. 4,195,907 to Zamja et al entitled “Light conducting Fibers” teaches the use of permanently bubbled optical fibers which can be used to form script-like static displays. The bubbles do not move and cannot be switched on and off. This is a decorative signage patent.        U.S. Pat. No. 4,422,719 to Orcutt entitled “Optical Distribution System Including Light Guide”. This patent is very similar to the above U.S. Pat. No. 4,195,907.        U.S. Pat. No. 4,466,697 to Daniel entitled “Light Dispersive Optical Lightpipes And Method Of Making Same”. This is again very similar to both U.S. Pat. No. 4,195,907 and U.S. Pat. No. 4,422,719.e) Unique Displays Fitting no Other Description        
The following references are cited and described:                U.S. Pat. No. 4,023,158 to Corcoran entitled “Real Three Dimensional Display Arrangement”. While this invention is creative, we seriously question its workability, as taught, as evidenced first by the lack of even a crude prototype or product after many years. We are familiar with optical levitation of particulates in a vacuum or gas (as well as in a liquid by optical laser tweezers). We are not aware of any demonstration wherein particles could be picked up and levitated in a gas or vacuum without somehow externally mechanically aiding their getting into the proper beam potential-well location (horizontal, vertical, or both) to begin with. This lifting phase is far more challenging than the metastable levitation itself. Secondly, an ideal 3-D display will allow for multiple voxels along a given Z height (Z-scanline) to be activated. The taught laser approach is incapable of levitating multiple particles one above the other along a vertical Z-axis. Thirdly, in order to obtain high-frame rate video, one would have to be juggling millions of these particles simultaneously, all without interfering particles or interfering beams. It is obviously impossible given the massive beam-interference that would take place as well as the need to wait for particles to be raised or dropped. We doubt this can be done even slowly for even only one such particle along a substantial Z-axis. One would likely require feedback sensing from each and every such particle, which is a difficult task for even one particle, let alone millions of masking particles. The particle inertia would also require one to have a variable velocity as each particle is moved into place, in order to prevent positional overshoot and positional oscillations. This will greatly slow down the process. There would also be significant heating of the particles and of any gas as well as intraparticle laser scattering. Finally, the intraparticle collisions will develop interfering dust as well as variable particle mass and reflectance/absorption. On a positive note, a levitated particle provides all the benefits of a screen without the screen (except at the illuminated points). From an optical display-performance point of view, if one could do what this patent seeks to at video rates, then the optical performance would be quite good (with the caveats such that one still cannot do multiple particles along a given Z-scanline nor reliably pick up the particles). We ask the reader to keep in mind the optical benefits of “suspended” particles. There are no interconnects inducing haze or screen-induced dead zones. That, to us, is the value in this reference, not the unworkable implementation it teaches. Particles can make optically great voxels of inherently good contrast and brightness. However, any scheme that requires that each and every particle be individually custom optically-levitated (feedback involved) is untenable in terms of frame rate, cost, and complexity. Even without feedback it is probably untenable. Nobody has demonstrated the full lift-levitate-drop cycle on a purely optical basis without external aids even for one particle to our knowledge.        U.S. Pat. No. 4,640,592 to Nishimura et al entitled “Optical Display Utilizing Thermally Formed Bubble In A Liquid Core Waveguide”. Essentially, these inventors build upon the prior art decorative static bubbled fibers and make new versions wherein the bubbles are not static but dynamic. By preparing a 2-D matrix of juxtaposed fibers with liquid bubbling capability, they create a 2-D dynamic display. They utilize the bubbles to redirect optical illumination into the viewer's eyes. The bubbles can most easily be formed by the use of a thin film heater that transfers heat into the adjacent or underlying liquid-filled fiber or channel. It is apparent that this approach should work at least in some modest manner in two dimensions, although no prototype is shown. There are, however, some issues which are not discussed in detail that could nullify or minimize the potential workability for reasonable frame rates. One needs to think about where the displaced fluid goes when one forms a bubble. The case of a lone bubble is quite different from 1,000 adjacent formed bubbles in terms of how much fluid needs to be displaced to somewhere and how quickly. Unless one provides a local displacement region at each bubble, then when one has multiple bubbles, one will have unique higher displacement demands controlling the time it takes to form (or make disappear) such bubble strings. This is not taught. Also, this technology will consume higher power compared to LCD and OLED technologies. We believe that the inventors minimized the abilities of conventional technologies and their expected (now-achieved) progress. These technologies have progressed greatly to high resolution and low power. Thus, the inventors present an interesting clever mechanism apparently usable at least for slow frame rates, but whose advantages, if any, over many competing lower-cost more power-efficient 2-D technologies (LCD, OLED, plasma, etc.) is unclear. Three-dimensional applications are not taught, and it is already known by prior inventors that trying to stack fiber optic layers in three dimensions is very difficult if the stack needs to be transparent in the off-state despite electrodes, heaters, etc. Nobody has ever built a transparent stacked 3-D display, where “transparent” means that there is negligible fog-like or diffractive interference with the images being presented and viewed. In any event, this reference is strictly a flat two-dimensional display.        FogScreen Inc.—The FogScreen™ product and The Walk-Through Fog Screen, University of Tampere, Finland, 2003. This is a clever two-dimensional screen for projection of images—largely advertisements—per FogScreen's stated marketing approach. Essentially, a roughly defined layer of fog is injected and carried between two adjacent layers of clear laminar downward flowing air. The downward squeezing laminar films of clear air keep the sandwiched fog layer at a pseudo-stable but non-zero thickness. Their fog is created, apparently, utilizing known ultrasonic water nebulizer or atomizer techniques. If the fog is made dense enough or the fog layer thick enough, one can approach useful screen opacity. In the normal pictured mode of operation, the fog consists of rapidly evaporating water droplets which do not wet an inserted hand. All of the photos we have seen of this display reveal that it is best viewed from directly in front in a darkened room. This is clearly because the screen has a non-zero thickness and if viewed from any nonzero angle, the image is badly defocused because of the screen's finite extinction depth. Obviously, if one made a heavy enough fog density in a thin enough layer, one could approach a thin screen; however, we expect that this will result in water condensation and variable opacity vs. height. Note that the downward flow is at relatively slow speeds (feet per second) in keeping with known laminar flow regimes. Thus, the technology is limited to static screens upon which static or video imagery is projected and which is viewed in the dark from directly in front (or from directly in back). The fog droplets are clearly and unalterably randomly located in space consistent with the definition of a fog. The core of the reference is the laminar-flow squeezed fog layer sandwich. The related art of visualizing laser and light beams (not images) in fogs and mists constitutes an industry of its own. Therein, mineral oil and glycol dispersed fogs are produced for stage shows, rock-concerts etc. The FogScreen™ is clever and will probably be a market success for advertising and public entertainment applications. It is not usable for any serious graphic purpose requiring true volumetric 3-D and good (computer or graphics type) image quality, resolution or walk-around capability.f) Summary of the Prior Art for True 3-D Volumetric Displays        
In terms of contrast and brightness, the rotating screen based e-beam/phosphor systems pictured in the Blundell book reference and of the type described in the rotating active-screen art above perform very nicely despite the fact that they only show relatively low resolution stick-figures or wire-frames. However, these systems are mechanically complex, very bulky, fragile, and must be of a multigun nature, and require further e-beam calibration development as shown by the second Blundell patent cited. CRT volumetric displays are also slow, being spot scanners.
In terms of video frame rate, co-rotating image projector/screen systems and stacked LCD panel projection systems cited in the above prior art should perform moderately well. However, stacked LCD systems must be extremely heavy and inevitably have a fogged or hazy appearance due to their nontransparent interconnects and layered interfaces, and we have never seen an actual prototype of a stacked LCD 3-D display. Co-rotated screen approaches with high-bandwidth data paths, as embodied by the Actuality Systems, Inc. product, probably represent the best current solution. However they are still of foggy hazy appearance due to the translucent screen always being in lines of sight and always having a non-zero effect on stray light, of limited contrast requiring near-darkness, lack of occlusion, and significant noise. It is not possible to judge the actual achieved resolution of the Actuality Systems, Inc. product from the images provided, but it certainly is not of conventional 2-D CRT quality in terms of contrast or resolution.
We have cited a few patents describing aesthetic and amusement devices that utilize suspended or falling droplets of water or other particles as the passive or active “voxels”. We believe that such free-standing droplets or particles (solid, liquid or gas) offer the perfect voxel for a 3-D display from the optical viewpoint, provided that much much smaller particles can be better and much much faster spatially managed with much greater precision. The water related prior art has not succeeded here in obtaining the needed fusion frame rates nor reasonable resolutions akin to printing or 2-D displays because that was not their purpose and it was not possible anyway. None of those inventors claimed to try and do what the inventors herein have as their goal. This is a far more severe challenge than projecting upon a random slowly sinking fog in the FogScreen™ manner. The means of making and handling such droplet voxel entities in order to make a workable 3-D graphics display usable for applications beyond aesthetics and amusement has not yet been taught. Clearly, this is why such historical water-based 3-D or pseudo-3-D devices are characterized correctly as amusement or aesthetic devices by the inventors that conceived them. Despite this, they still provide aesthetic pleasure and amusement for those different markets.
Clearly, the generally available graphics electronics, optoelectronic components (digital light mirrors, solid-state lasers, super-bright LEDs, for example), firmware and software are progressing rapidly, as shown by the Actuality Systems, Inc. product, but the image quality is still obviously modest at best when viewed even under favorable darkness. Even the most basic low-end 2-D displays still have much much better image quality, partly because of their superior effective contrast, brightness and persistence and partly due to their better effective resolutions and lack of fog or haze. It is our intent, with the invention herein, to bring the technology of 3-D displays more on par with 2-D displays in terms of image quality. We believe that in order to do this, something fundamentally new is required.
We will discuss the subject of visual fusion herein repeatedly as it is an exceedingly useful and time-honored general approach to the visual integration of much displayable subject matter via, for example, the known mechanisms of visual/brain persistence. We stress, however, that one can use such persistence effects to not only depict moving objects, scenes and subject matter but in the dynamic refreshing of static objects, scenes or subject matter. Furthermore, some embodiments herein can display static displayable matter without such fusion-refreshment.